Perturbations of Markov Chains with Applications to Stochastic Games
نویسنده
چکیده
In this lecture we will review several topics that are extensively used in the study of n-player stochastic games. These tools were used in the proof of several results on non zero-sum stochastic games. Most of the results that are presented here appeared in Vieille (1997a,b), and some appeared in Solan (1998, 1999). The first main issue is Markov chains where the transition rule is a Puiseux probability distribution. We define the notion of communicating sets and induce a hierarchy on the collection of these sets. We then relate these concepts to stochastic games, and show several conditions that enable the players to control exit distributions from communicating sets.
منابع مشابه
Stochastic Dynamic Programming with Markov Chains for Optimal Sustainable Control of the Forest Sector with Continuous Cover Forestry
We present a stochastic dynamic programming approach with Markov chains for optimal control of the forest sector. The forest is managed via continuous cover forestry and the complete system is sustainable. Forest industry production, logistic solutions and harvest levels are optimized based on the sequentially revealed states of the markets. Adaptive full system optimization is necessary for co...
متن کاملStochastic bounds for a single server queue with general retrial times
We propose to use a mathematical method based on stochastic comparisons of Markov chains in order to derive performance indice bounds. The main goal of this paper is to investigate various monotonicity properties of a single server retrial queue with first-come-first-served (FCFS) orbit and general retrial times using the stochastic ordering techniques.
متن کاملMarkov Stationary Equilibria in Stochastic Supermodular Games with Imperfect Private and Public Information
We study a class of discounted, infinite horizon stochastic games with public and private signals and strategic complementarities. Using monotone operators defined on the function space of values and strategies (equipped with a product order), we prove existence of a stationary Markov–Nash equilibrium via constructive methods. In addition, we provide monotone comparative statics results for ord...
متن کاملFrom Markov Chains to Stochastic Games
Markov chains1 and Markov decision processes (MDPs) are special cases of stochastic games. Markov chains describe the dynamics of the states of a stochastic game where each player has a single action in each state. Similarly, the dynamics of the states of a stochastic game form a Markov chain whenever the players’ strategies are stationary. Markov decision processes are stochastic games with a ...
متن کاملA Perturbation Theory for Ergodic Markov Chains and Application to Numerical Approximations
Perturbations to Markov chains and Markov processes are considered. The unperturbed problem is assumed to be geometrically ergodic in the sense usually established through the use of Foster–Lyapunov drift conditions. The perturbations are assumed to be uniform, in a weak sense, on bounded time intervals. The long-time behavior of the perturbed chain is studied. Applications are given to numeric...
متن کامل